NVIDIA, AMD, and Intel now offer workstation GPUs with up to 32 GB VRAM for local AI inference, but VRAM capacity and memory bandwidth matter more than TOPS for running large language models. The RTX …
ARCQuant, a new framework for Large Language Model inference, uses the NVFP4 numerical format to achieve up to 3x speedup on GPUs while maintaining accuracy comparable to full-precision baselines. The…
NVIDIA announced that its Confidential Computing technology for Blackwell GPUs achieves up to 98% of the inference performance of non-secure solutions, enabling hardware-rooted AI security without sig…
NVIDIA introduced BEVPoolV3, an optimized bird's-eye-view pooling operator for autonomous vehicles and robotics, reducing latency on RTX GPUs by removing redundant scatter traffic and precomputing ind…
A 744-billion-parameter GLM-5.2 model was served at ~30 tokens per second across six prosumer Blackwell GPUs in six US states over a wide-area network using pipeline parallelism and speculative decodi…
Nvidia, in partnership with Span and PulteGroup, launched the XFRA initiative to install mini AI data centers in residential backyards, using surplus household electricity to power compact computing n…